Low Rank and Sparse Fourier Structure in Recurrent Networks Trained on Modular Addition Akshay Rangamani Dept. of Data Science New Jersey Institute of T echnology Newark, NJ, USA akshay.rangamani@njit.edu Abstract --Modular addition tasks serve as a useful test bed for observing empirical phenomena in deep learning, including the phenomenon of grokking. Prior work has shown that one-layer transformer architectures learn Fourier Multiplication circuits to solve modular addition tasks. In this paper, we show that Recurrent Neural Networks (RNNs) trained on modular addition tasks also use a Fourier Multiplication strategy. We identify low rank structures in the model weights, and attribute model components to specific Fourier frequencies, resulting in a sparse representation in the Fourier space. We also show empirically that the RNN is robust to removing individual frequencies, while the performance degrades drastically as more frequencies are ablated from the model.

A hallmark of intelligence is the ability to flesh out underspecified situations using "common sense." We propose to extract that common sense from large language models (LLMs), in a form that can feed into probabilistic inference. We focus our investigation on $\textit{guesstimation}$ questions such as "How much are Airbnb listings in Newark, NJ?" Formulating a sensible answer without access to data requires drawing on, and integrating, bits of common knowledge about how $\texttt{Price}$ and $\texttt{Location}$ may relate to other variables, such as $\texttt{Property Type}$. Our framework answers such a question by synthesizing an $\textit{ad hoc}$ probabilistic model. First we prompt an LLM to propose a set of random variables relevant to the question, followed by moment constraints on their joint distribution. We then optimize the joint distribution $p$ within a log-linear family to maximize the overall constraint satisfaction. Our experiments show that LLMs can successfully be prompted to propose reasonable variables, and while the proposed numerical constraints can be noisy, jointly optimizing for their satisfaction reconciles them. When evaluated on probabilistic questions derived from three real-world tabular datasets, we find that our framework performs comparably to a direct prompting baseline in terms of total variation distance from the dataset distribution, and is similarly robust to noise.

Mehmet Aytekin, 28, left, checks his cell phone while waiting to board his United Airlines flight to Newark, N.J. at O'Hare International Airport on Jan. 3, 2020. Amid high costs and controversies surrounding college education – coupled with the threat that artificial intelligence poses on certain white-collar jobs – much of Gen Z is leaning toward pursuing trade schools and blue-collar jobs with that tech gap in mind. IN ITS'PRIME': Amazon.com reported record first-quarter sales as the AI boom powered growth in its cloud-computing unit, helping the company continue to shake off last year's post-pandemic slump. FUTURE'S NOT SET: Policymakers should not reference or rely on fictional scenarios as reasons to regulate AI. Otherwise, America risks losing its global lead on AI and American citizens could never realize the full benefits of the technology.

We help the world see new possibilities and inspire change for better tomorrows. Our analytic solutions bridge content, data, and analytics to help business, people, and society become stronger, more resilient, and sustainable. The Data Engineering and Analytics Lab (DEAL) is a team of technical actuaries responsible for the design and implementation of our core statistical data-systems including data ingestion, data integration, data transformation, data analysis, and analytic dataset construction. We're an innovation group that is charged with visualizing the future of our organization's operations and leveraging our expertise in data, technology, P&C insurance, and process optimization to provide a first-class analytics environment to our data-collection, data-management, actuarial, and data-analytics colleagues. The DEAL team is looking to hire an experienced Lead ETL Data Engineer, ideally having a good combination of an analytical/innovative mindset, technical aptitude, business accumen, communication skills, and a passion for mentoring.

Artificial intelligence (AI) offers government utilities a transformative opportunity to improve public service, update outdated processes and centralize data. In particular, water utilities can use AI to make timely repairs and adjustments in a way that poses fewer inconveniences for citizens. One way to facilitate such AI modernization within water utilities is through public-private partnerships. Cities like Tucson, Ariz., and Newark, N.J., are leading by example. Tucson teamed up with VODA.ai in August 2020 to bring AI technology to its water infrastructure systems. Prior to using AI technology as a tool, the city relied on pipe-break history and human judgment to drive maintenance projects. Newark Water and Sewer also adopted AI to make more informed decisions and to take a more proactive approach to improve water infrastructure, according to Tiffany Stewart, assistant director of the utility. The department is using two artificial intelligence systems for unique purposes.

Substitute model black box attacks can create adversarial examples for a target model just by accessing its output labels. This poses a major challenge to machine learning models in practice, particularly in security sensitive applications. The 01 loss model is known to be more robust to outliers and noise than convex models that are typically used in practice. Motivated by these properties we present 01 loss linear and 01 loss dual layer neural network models as a defense against transfer based substitute model black box attacks. We compare the accuracy of adversarial examples from substitute model black box attacks targeting our 01 loss models and their convex counterparts for binary classification on popular image benchmarks. Our 01 loss dual layer neural network has an adversarial accuracy of 66.2%, 58%, 60.5%, and 57% on MNIST, CIFAR10, STL10, and ImageNet respectively whereas the sigmoid activated logistic loss counterpart has accuracies of 63.5%, 19.3%, 14.9%, and 27.6%. Except for MNIST the convex counterparts have substantially lower adversarial accuracies. We show practical applications of our models to deter traffic sign and facial recognition adversarial attacks. On GTSRB street sign and CelebA facial detection our 01 loss network has 34.6% and 37.1% adversarial accuracy respectively whereas the convex logistic counterpart has accuracy 24% and 1.9%. Finally we show that our 01 loss network can attain robustness on par with simple convolutional neural networks and much higher than its convex counterpart even when attacked with a convolutional network substitute model. Our work shows that 01 loss models offer a powerful defense against substitute model black box attacks.

In late 2017, AB InBev, the Belgian giant behind Budweiser and other beers, began adding a little artificial intelligence to its brewing recipe. Using data collected from a brewery in Newark, New Jersey, the company developed an AI algorithm to predict potential problems with the filtration process used to remove impurities from beer. Paul Silverman, who runs the New Jersey Beer Company, a small operation not far from the AB InBev brewery, says his team isn't even using computers, let alone AI. "We sit around tasting beer and thinking about what to make next," he says. The divide between the two breweries highlights the pace at which AI is being adopted by US companies. With so much hype around artificial intelligence, you might imagine that it's everywhere.

These shifts will require new executives, new jobs, and new responsibilities. Paul Daugherty, chief technology and innovation officer at Accenture, sees three myths surrounding artificial intelligence: Robots are coming for us, machines will take our jobs, and current approaches to business processes will still apply. The three myths represent "conventional changes to linear processes," he said. The reality is more transformative. An example: Newark, New Jersey-based AeroFarms grows seeds indoors without soil or sunlight.

Paul Daugherty, chief technology and innovation officer at Accenture, sees three myths surrounding artificial intelligence: Robots are coming for us, machines will take our jobs, and current approaches to business processes will still apply. The three myths represent "conventional changes to linear processes," he said. The reality is more transformative. An example: Newark, New Jersey-based AeroFarms grows seeds indoors without soil or sunlight. Seeds are harvested in less than three weeks and the process requires 95 percent less water than conventional farming methods.

There will be 600,000 commercial drone aircraft operating in the U.S. within the year as the result of new safety rules that opened the skies to them on Monday, according to a Federal Aviation Administration estimate. The rules governing the operation of small commercial drones were designed to protect safety without stifling innovation, FAA Administrator Michael Huerta told a news conference. Commercial operators initially complained that the new rules would be too rigid. FILE - In this May 21, 2015 file photo, Federal Aviation Administration Administrator Michael Huerta, speaks during a news conference at Newark Liberty International Airport in Newark, N.J. Federal aviation officials estimate there will be 600,000 commercial drone aircraft operating in the U.S. within the year as the result of new safety rules that went into effect on Monday, Aug. 29, 2016.